Apparatus and method for displaying AR object

ABSTRACT

A method and electronic device for displaying an augmented reality (AR) object recognized by the electronic device are provided. The electronic device includes a display, a memory including stored instructions, and at least one processor. The at least one processor is coupled with the memory and the display. The at least one processor is configured to execute the stored instructions so as to identify a first object on a preview acquired from a first camera, and display a first AR object related with the first object on the preview, and in response to obtaining an input for converting from the first camera to a second camera, store information about the first AR object, and display a second AR object which is related with a second object comprised in a preview of the second camera and the information about the first AR object, on the preview of the second camera.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) of a Korean patent application number 10-2017-0123973, filed onSep. 26, 2017, in the Korean Intellectual Property Office, thedisclosure of which is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to a method and electronic device for displayingan augmented reality (AR) object recognized by the electronic device.

2. Description of Related Art

With the development of electronic device technologies, electronicdevices can provide various experiences to users of the electronicdevices. For example, the electronic device can provide augmentedreality (AR) of adding virtual information to the real thing and showingthe same. The electronic device can recognize the real thing through acamera, and can display information (e.g., a virtual object) about AR ona preview of the recognized thing. This AR can be provided to the userthrough a game application or a photo application. For example, the gameapplication can recognize the real thing through the camera, and displaya related game character. For another example, the photo application canrecognize a face through the camera, and display a virtual object aroundthe face.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, an electronic device isprovided. The electronic device includes a display, a memory comprisinginstructions stored thereon, and at least one processor coupled with thememory and the display, wherein the at least one processor is configuredto execute the stored instructions so as to identify a first object on apreview acquired from a first camera, display a first augmented reality(AR) object related with the first object on the preview, in response toobtaining an input for converting from the first camera to a secondcamera, store information about the first AR object and display a secondAR object which is related with a second object comprised in a previewof the second camera and the information about the first AR object, onthe preview of the second camera.

In accordance with another aspect of the disclosure, a method for anelectronic device is provided. The method includes identifying a firstobject on a preview acquired from a first camera, displaying a first ARobject related with the first object on the preview, in response toobtaining an input for converting from the first camera to a secondcamera, storing information about the first AR object and displaying asecond AR object which is related with a second object comprised in apreview of the second camera and the information about the first ARobject, on the preview of the second camera.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram of an electronic device within a networkenvironment according to various embodiments of the disclosure;

FIG. 2 is a block diagram of a display module according to variousembodiments of the disclosure;

FIG. 3 is a block diagram of a camera module according to variousembodiments of the disclosure;

FIG. 4 illustrates an example of a signal flow of an electronic devicefor displaying an augmented reality (AR) object according to variousembodiments of the disclosure;

FIG. 5 illustrates an example of an operation of an electronic devicefor displaying an AR object according to various embodiments of thedisclosure;

FIG. 6 illustrates another example of an operation of an electronicdevice for displaying an AR object according to various embodiments ofthe disclosure;

FIG. 7 illustrates an example of a user interface (UI) for displaying anAR object according to various embodiments of the disclosure;

FIG. 8 illustrates another example of a UI for displaying an AR objectaccording to various embodiments of the disclosure;

FIG. 9 illustrates a further example of a UI for displaying an AR objectaccording to various embodiments of the disclosure;

FIG. 10 illustrates a yet another example of a UI for displaying an ARobject according to various embodiments of the disclosure;

FIG. 11 illustrates an example of an operation of an electronic devicefor displaying a final AR object according to various embodiments of thedisclosure;

FIG. 12 illustrates an example of a UI for displaying a final AR objectaccording to various embodiments of the disclosure;

FIG. 13 illustrates another example of a UI for displaying a final ARobject according to various embodiments of the disclosure;

FIG. 14 illustrates an example of an operation of an electronic devicefor displaying a recommendation AR object according to variousembodiments of the disclosure;

FIG. 15 illustrates another example of an operation of an electronicdevice for displaying a recommendation AR object according to variousembodiments of the disclosure;

FIG. 16 illustrates an example of a UI for displaying a recommendationAR object according to various embodiments of the disclosure;

FIG. 17 illustrates an example of a signal flow of an electronic devicefor displaying an AR object related with another electronic deviceaccording to various embodiments of the disclosure;

FIG. 18 illustrates an example of a UI of an electronic device fordisplaying an AR object related with another electronic device accordingto various embodiments of the disclosure; and

FIG. 19 illustrates another example of a UI of an electronic device fordisplaying an AR object related with another electronic device accordingto various embodiments of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

FIG. 1 is a block diagram illustrating an electronic device 101 in anetwork environment 100 according to various embodiments of thedisclosure.

Referring to FIG. 1, the electronic device 101 in the networkenvironment 100 may communicate with an electronic device 102 via afirst network 198 (e.g., a short-range wireless communication network),or an electronic device 104 or a server 108 via a second network 199(e.g., a long-range wireless communication network). According to anembodiment, the electronic device 101 may communicate with theelectronic device 104 via the server 108. According to an embodiment,the electronic device 101 may include a processor 120, memory 130, aninput device 150, a sound output device 155, a display device 160, anaudio module 170, a sensor module 176, an interface 177, a haptic module179, a camera module 180, a power management module 188, a battery 189,a communication module 190, a subscriber identification module (SIM)196, or an antenna module 197. In some embodiments, at least one (e.g.,the display device 160 or the camera module 180) of the components maybe omitted from the electronic device 101, or one or more othercomponents may be added in the electronic device 101. In someembodiments, some of the components may be implemented as singleintegrated circuitry. For example, the sensor module 176 (e.g., afingerprint sensor, an iris sensor, or an illuminance sensor) may beimplemented as embedded in the display device 160 (e.g., a display).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 101 coupled with theprocessor 120, and may perform various data processing or computation.According to one embodiment, as at least part of the data processing orcomputation, the processor 120 may load a command or data received fromanother component (e.g., the sensor module 176 or the communicationmodule 190) in volatile memory 132, process the command or the datastored in the volatile memory 132, and store resulting data innon-volatile memory 134. According to an embodiment, the processor 120may include a main processor 121 (e.g., a central processing unit (CPU)or an application processor (AP)), and an auxiliary processor 123 (e.g.,a graphics processing unit (GPU), an image signal processor (ISP), asensor hub processor, or a communication processor (CP)) that isoperable independently from, or in conjunction with, the main processor121. Additionally or alternatively, the auxiliary processor 123 may beadapted to consume less power than the main processor 121, or to bespecific to a specified function. The auxiliary processor 123 may beimplemented as separate from, or as part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display device 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an ISP or a CP) may beimplemented as part of another component (e.g., the camera module 180 orthe communication module 190) functionally related to the auxiliaryprocessor 123.

The memory 130 may store various data used by at least one component(e.g., the processor 120 or the sensor module 176) of the electronicdevice 101. The various data may include, for example, software (e.g.,the program 140) and input data or output data for a command relatedthereto. The memory 130 may include the volatile memory 132 or thenon-volatile memory 134.

The program 140 may be stored in the memory 130 as software, and mayinclude, for example, an operating system (OS) 142, middleware 144, oran application 146.

The input device 150 may receive a command or data to be used by anothercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputdevice 150 may include, for example, a microphone, a mouse, or akeyboard.

The sound output device 155 may output sound signals to the outside ofthe electronic device 101. The sound output device 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for incoming calls. According to an embodiment, the receivermay be implemented as separate from, or as part of the speaker.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 160 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 170 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 170 may obtainthe sound via the input device 150, or output the sound via the soundoutput device 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector),

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, ISPs, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to one embodiment, the power managementmodule 188 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more CPs that are operableindependently from the processor 120 (e.g., the AP) and supports adirect (e.g., wired) communication or a wireless communication.According to an embodiment, the communication module 190 may include awireless communication module 192 (e.g., a cellular communicationmodule, a short-range wireless communication module, or a globalnavigation satellite system (GNSS) communication module) or a wiredcommunication module 194 (e.g., a local area network (LAN) communicationmodule or a power line communication (PLC) module). A corresponding oneof these communication modules may communicate with the externalelectronic device via the first network 198 (e.g., a short-rangecommunication network, such as Bluetooth™, Wi-Fi direct, or infrareddata association (IrDA)) or the second network 199 (e.g., a long-rangecommunication network, such as a cellular network, the Internet, or acomputer network (e.g., local area network (LAN) or wide area network(WAN)). These various types of communication modules may be implementedas a single component (e.g., a single chip), or may be implemented asmulti components (e.g., multi chips) separate from each other. Thewireless communication module 192 may identify and authenticate theelectronic device 101 in a communication network, such as the firstnetwork 198 or the second network 199, using subscriber information(e.g., international mobile subscriber identity (IMSI)) stored in theSIM 196.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include one or more antennas, and, therefrom, at least oneantenna appropriate for a communication scheme used in the communicationnetwork, such as the first network 198 or the second network 199, may beselected, for example, by the communication module 190 (e.g., thewireless communication module 192). The signal or the power may then betransmitted or received between the communication module 190 and theexternal electronic device via the selected at least one antenna.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the electronic devices 102 and 104 may be a device of a same type as,or a different type, from the electronic device 101. According to anembodiment, all or some of operations to be executed at the electronicdevice 101 may be executed at one or more of the external electronicdevices 102, 104, or 108. For example, if the electronic device 101should perform a function or a service automatically, or in response toa request from a user or another device, the electronic device 101,instead of, or in addition to, executing the function or the service,may request the one or more external electronic devices to perform atleast part of the function or the service. The one or more externalelectronic devices receiving the request may perform the at least partof the function or the service requested, or an additional function oran additional service related to the request, and transfer an outcome ofthe performing to the electronic device 101. The electronic device 101may provide the outcome, with or without further processing of theoutcome, as at least part of a reply to the request. To that end, acloud computing, distributed computing, or client-server computingtechnology may be used, for example.

FIG. 2 is a block diagram 200 illustrating the display device 160according to various embodiments of the disclosure.

Referring to FIG. 2, the display device 160 may include a display 210and a display driver integrated circuit (DDI) 230 to control the display210. The DDI 230 may include an interface module 231, memory 233 (e.g.,buffer memory), an image processing module 235, or a mapping module 237.The DDI 230 may receive image information that contains image data or animage control signal corresponding to a command to control the imagedata from another component of the electronic device 101 via theinterface module 231. For example, according to an embodiment, the imageinformation may be received from the processor 120 (e.g., the mainprocessor 121 (e.g., an AP)) or the auxiliary processor 123 (e.g., aGPU) operated independently from the function of the main processor 121.The DDI 230 may communicate, for example, with touch circuitry 250 orthe sensor module 176 via the interface module 231. The DDI 230 may alsostore at least part of the received image information in the memory 233,for example, on a frame by frame basis.

The image processing module 235 may perform pre-processing orpost-processing (e.g., adjustment of resolution, brightness, or size)with respect to at least part of the image data. According to anembodiment, the pre-processing or post-processing may be performed, forexample, based at least in part on one or more characteristics of theimage data or one or more characteristics of the display 210.

The mapping module 237 may generate a voltage value or a current valuecorresponding to the image data pre-processed or post-processed by theimage processing module 235. According to an embodiment, the generatingof the voltage value or current value may be performed, for example,based at least in part on one or more attributes of the pixels (e.g., anarray, such as a red-green-blue (RGB) stripe or a pentile structure, ofthe pixels, or the size of each subpixel). At least some pixels of thedisplay 210 may be driven, for example, based at least in part on thevoltage value or the current value such that visual information (e.g., atext, an image, or an icon) corresponding to the image data may bedisplayed via the display 210.

According to an embodiment, the display device 160 may further includethe touch circuitry 250. The touch circuitry 250 may include a touchsensor 251 and a touch sensor IC 253 to control the touch sensor 251.The touch sensor IC 253 may control the touch sensor 251 to sense atouch input or a hovering input with respect to a certain position onthe display 210. To achieve this, for example, the touch sensor 251 maydetect (e.g., measure) a change in a signal (e.g., a voltage, a quantityof light, a resistance, or a quantity of one or more electric charges)corresponding to the certain position on the display 210. The touchcircuitry 250 may provide input information (e.g., a position, an area,a pressure, or a time) indicative of the touch input or the hoveringinput detected via the touch sensor 251 to the processor 120. Accordingto an embodiment, at least part (e.g., the touch sensor IC 253) of thetouch circuitry 250 may be formed as part of the display 210 or the DDI230, or as part of another component (e.g., the auxiliary processor 123)disposed outside the display device 160.

According to an embodiment, the display device 160 may further includeat least one sensor (e.g., a fingerprint sensor, an iris sensor, apressure sensor, or an illuminance sensor) of the sensor module 176 or acontrol circuit for the at least one sensor. In such a case, the atleast one sensor or the control circuit for the at least one sensor maybe embedded in one portion of a component (e.g., the display 210, theDDI 230, or the touch circuitry 250)) of the display device 160. Forexample, when the sensor module 176 embedded in the display device 160includes a biometric sensor (e.g., a fingerprint sensor), the biometricsensor may obtain biometric information (e.g., a fingerprint image)corresponding to a touch input received via a portion of the display210. As another example, when the sensor module 176 embedded in thedisplay device 160 includes a pressure sensor, the pressure sensor mayobtain pressure information corresponding to a touch input received viaa partial or whole area of the display 210. According to an embodiment,the touch sensor 251 or the sensor module 176 may be disposed betweenpixels in a pixel layer of the display 210, or over or under the pixellayer.

FIG. 3 is a block diagram 300 illustrating the camera module 180according to various embodiments of the disclosure.

Referring to FIG. 3, the camera module 180 may include a lens assembly310, a flash 320, an image sensor 330, an image stabilizer 340, memory350 (e.g., buffer memory), or an ISP 360. The lens assembly 310 maycollect light emitted or reflected from an object whose image is to betaken. The lens assembly 310 may include one or more lenses. Accordingto an embodiment, the camera module 180 may include a plurality of lensassemblies 310. In such a case, the camera module 180 may form, forexample, a dual camera, a 360-degree camera, or a spherical camera. Someof the plurality of lens assemblies 310 may have the same lens attribute(e.g., view angle, focal length, auto-focusing, f number, or opticalzoom), or at least one lens assembly may have one or more lensattributes different from those of another lens assembly. The lensassembly 310 may include, for example, a wide-angle lens or a telephotolens.

The flash 320 may emit light that is used to reinforce light reflectedfrom an object. According to an embodiment, the flash 320 may includeone or more light emitting diodes (LEDs) (e.g., a RGB LED, a white LED,an IR LED, or an ultraviolet (UV) LED) or a xenon lamp. The image sensor330 may obtain an image corresponding to an object by converting lightemitted or reflected from the object and transmitted via the lensassembly 310 into an electrical signal. According to an embodiment, theimage sensor 330 may include one selected from image sensors havingdifferent attributes, such as an RGB sensor, a black-and-white (BW)sensor, an IR sensor, or a UV sensor, a plurality of image sensorshaving the same attribute, or a plurality of image sensors havingdifferent attributes. Each image sensor included in the image sensor 330may be implemented using, for example, a charged coupled device (CCD)sensor or a complementary metal oxide semiconductor (CMOS) sensor.

The image stabilizer 340 may move the image sensor 330 or at least onelens included in the lens assembly 310 in a particular direction, orcontrol an operational attribute (e.g., adjust the read-out timing) ofthe image sensor 330 in response to the movement of the camera module180 or the electronic device 101 including the camera module 180. Thisallows compensating for at least part of a negative effect (e.g., imageblurring) by the movement on an image being captured. According to anembodiment, the image stabilizer 340 may sense such a movement by thecamera module 180 or the electronic device 101 using a gyro sensor (notshown) or an acceleration sensor (not shown) disposed inside or outsidethe camera module 180. According to an embodiment, the image stabilizer340 may be implemented, for example, as an optical image stabilizer.

The memory 350 may store, at least temporarily, at least part of animage obtained via the image sensor 330 for a subsequent imageprocessing task. For example, if image capturing is delayed due toshutter lag or multiple images are quickly captured, a raw imageobtained (e.g., a Bayer-patterned image, a high-resolution image) may bestored in the memory 350, and its corresponding copy image (e.g., alow-resolution image) may be previewed via the display device 160.Thereafter, if a specified condition is met (e.g., by a user's input orsystem command), at least part of the raw image stored in the memory 350may be obtained and processed, for example, by the ISP 360. According toan embodiment, the memory 350 may be configured as at least part of thememory 130 or as a separate memory that is operated independently fromthe memory 130.

The ISP 360 may perform one or more image processing with respect to animage obtained via the image sensor 330 or an image stored in the memory350. The one or more image processing may include, for example, depthmap generation, three-dimensional (3D) modeling, panorama generation,feature point extraction, image synthesizing, or image compensation(e.g., noise reduction, resolution adjustment, brightness adjustment,blurring, sharpening, or softening). Additionally or alternatively, theISP 360 may perform control (e.g., exposure time control or read-outtiming control) with respect to at least one (e.g., the image sensor330) of the components included in the camera module 180. An imageprocessed by the ISP 360 may be stored back in the memory 350 forfurther processing, or may be provided to an external component (e.g.,the memory 130, the display device 160, the electronic device 102, theelectronic device 104, or the server 108) outside the camera module 180.According to an embodiment, the ISP 360 may be configured as at leastpart of the processor 120, or as a separate processor that is operatedindependently from the processor 120. If the ISP 360 is configured as aseparate processor from the processor 120, at least one image processedby the ISP 360 may be displayed, by the processor 120, via the displaydevice 160 as it is or after being further processed.

According to an embodiment, the electronic device 101 may include aplurality of camera modules 180 having different attributes orfunctions. In such a case, at least one of the plurality of cameramodules 180 may form, for example, a wide-angle camera and at leastanother of the plurality of camera modules 180 may form a telephotocamera. Similarly, at least one of the plurality of camera modules 180may form, for example, a front camera and at least another of theplurality of camera modules 180 may form a rear camera.

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smart phone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

It should be appreciated that various embodiments of the disclosure andthe terms used therein are not intended to limit the technologicalfeatures set forth herein to particular embodiments and include variouschanges, equivalents, or replacements for a corresponding embodiment.With regard to the description of the drawings, similar referencenumerals may be used to refer to similar or related elements. It is tobe understood that a singular form of a noun corresponding to an itemmay include one or more of the things, unless the relevant contextclearly indicates otherwise. As used herein, each of such phrases as “Aor B,” “at least one of A and B,” “at least one of A or B,” “A, B, orC,” “at least one of A, B, and C,” and “at least one of A, B, or C,” mayinclude all possible combinations of the items enumerated together in acorresponding one of the phrases. As used herein, such terms as “1st”and “2nd,” or “first” and “second” may be used to simply distinguish acorresponding component from another, and does not limit the componentsin other aspect (e.g., importance or order). It is to be understood thatif an element (e.g., a first element) is referred to, with or withoutthe term “operatively” or “communicatively”, as “coupled with,” “coupledto,” “connected with,” or “connected to” another element (e.g., a secondelement), it means that the element may be coupled with the otherelement directly (e.g., wiredly), wirelessly, or via a third element.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic,” “logic block,” “part,” or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, the module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 140) including one or more instructions that arestored in a storage medium (e.g., internal memory 136 or external memory138) that is readable by a machine (e.g., the electronic device 101).For example, a processor (e.g., the processor 120) of the machine (e.g.,the electronic device 101) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a complier or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program product may be traded as a product betweena seller and a buyer. The computer program product may be distributed inthe form of a machine-readable storage medium (e.g., compact disc readonly memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded)online via an application store (e.g., Play Store™), or between two userdevices (e.g., smart phones) directly. If distributed online, at leastpart of the computer program product may be temporarily generated or atleast temporarily stored in the machine-readable storage medium, such asmemory of the manufacturer's server, a server of the application store,or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities. According to various embodiments, one or more ofthe above-described components may be omitted, or one or more othercomponents may be added. Alternatively or additionally, a plurality ofcomponents (e.g., modules or programs) may be integrated into a singlecomponent. In such a case, according to various embodiments, theintegrated component may still perform one or more functions of each ofthe plurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various embodiments, operations performedby the module, the program, or another component may be carried outsequentially, in parallel, repeatedly, or heuristically, or one or moreof the operations may be executed in a different order or omitted, orone or more other operations may be added.

FIG. 4 illustrates an example of a signal flow of an electronic devicefor displaying an augmented reality (AR) object according to variousembodiments of the disclosure.

For description convenience's sake, exemplified are a term, aconstituent element, etc. denoting an AR object (e.g., a final AR objectand/or a recommendation AR object) used in the following description.Accordingly, the disclosure is not limited to the terms described later,and may use other terms having equivalent technological meanings.

In various embodiments, the electronic device 101 may include theprocessor 120, the display device 160, a first camera 401 or a secondcamera 402. The processor 120, the display device 160, the first camera401 or the second camera 402 may be coupled with one another. Forexample, at least some of the processor 120, the display device 160, thefirst camera 401 or the second camera 402 may be directly connected toone another. For another example, at least some of the processor 120,the display device 160, the first camera 401 or the second camera 402may be indirectly connected to one another through another device orcircuitry.

In various embodiments, the first camera may be a rear camera, and thesecond camera may be a front camera. However, the disclosure is notlimited to the aforementioned example, and may identify any one of twocameras which are included in or connected with the electronic device101, as the first camera, and identify the other one as the secondcamera.

In various embodiments, the processor 120 may be configured to executeinstructions stored in the memory 130 in order to execute a function ofat least one constituent element included in the electronic device 101.For example, the processor 120 may be configured to execute the storedinstructions so as to enable the first camera 401 or the second camera402 to acquire an image.

In operation 403, the processor 120 may obtain an input for execution ofan application installed (or stored) in the electronic device 101. Theapplication may be for displaying an AR object. The input for executionof the application may be an input that a user of the electronic device101 touches an application icon displayed on the display 210. The inputof touching the application icon may include at least one of a tap, adouble tap, a drag and drop, or a long tap, and is not limited to theaforementioned example. By obtaining the input to the application, theprocessor 120 may execute the application.

In various embodiments, the application may be an application programmedto show AR, i.e., an application based on AR. For example, theapplication may be an AR based camera application or an AR based gameapplication. Here, the AR may refer an image (or picture) provided by atechnology of overlaying and showing real images or 3D virtual images asone image.

In operation 405, the processor 120 may transmit a signal for executingthe first camera 401 to the first camera 401. In response to theapplication being executed, the processor 120 may transmit a signal forexecution of the first camera 401 to the first camera 401.

In operation 407, the first camera 401 may be executed. In response toreceiving the signal of operation 405, the first camera 401 may turn On.

In operation 409, the first camera 401 may recognize (or acquire) apreview including a first object. The first camera 401 is executed inoperation 407, whereby the first camera 401 may acquire an image. Theacquired image may include the first object, and the first object may bea previously given object or thing. The acquired image may be displayedin the form of a preview for previewing scenes recognized through thefirst camera 401 within a user interface (UI) of the electronic device101.

In various embodiments, the first object may be a target object forshowing AR, and information about the first object may be stored in aserver related with AR (or an AR based application). The informationabout the first object may, for example, be at least one of a location,color, shape, form or size of the first object.

Though not illustrated, in various embodiments, the electronic device101 may be connected with the server related with the application. Thefirst object may be previously given, and the information about thefirst object may have been stored in the memory 130 or the server. Forexample, the information about the first object may be stored in theserver, and an image acquired based on a connection between theelectronic device 101 and the server may be identified, whereby thefirst object may be identified.

In operation 411, the first camera 401 may transmit information about arecognized preview to the processor 120. The first camera 401 maycontinuously recognize the preview, thereby refining the recognizedpreview, and may transmit information about the refined preview to theprocessor 120 in real time.

In operation 413, the processor 120 may transmit a signal for executionof the second camera 402 to the second camera 402. In response to theapplication being executed, the processor 120 may transmit a signal forexecution of the first camera 401 to the first camera 401.

In operation 415, the second camera 402 may be executed. In response toreceiving the signal of operation 413, the second camera 402 may turnOn.

In operation 417, the second camera 402 may recognize (or acquire) apreview including a second object. The second object may be an object orthing included within the preview recognized through the second camera402. The second object may be previously given, and information aboutthe second object may have been stored in the memory 130 or the server.For example, the information about the second object may be stored inthe server, and an image acquired based on a connection between theelectronic device and the server may be identified, whereby the secondobject may be identified.

Though not illustrated, in various embodiments, the electronic device101 may be connected with the server related with the application. Thesecond object may be previously given, and the information about thesecond object may have been stored in the memory 130 or the server ofthe application. For example, the information about the second objectmay be stored in the server, and an image acquired based on theconnection between the electronic device and the server may beidentified, whereby the second object may be identified.

In operation 419, the second camera 402 may transmit information aboutthe recognized preview to the processor 120. The second camera 402 maycontinuously recognize the preview, thereby refining the recognizedpreview, and may transmit information about the refined preview to theprocessor 120 in real time.

In operation 421, the processor 120 may identify, as the first camera401, a camera that will display a preview. In various embodiments, theprocessor 120 may identify whether it has ever used the first camera 401formerly, in relation with execution of an application. In response tohaving ever used the first camera 401 formerly, the processor 120 mayidentify, as the first camera 401, the camera that will display thepreview. Or, in response to having ever used the second camera 402formerly, the processor 120 may identify, as the second camera 402, thecamera that will display the preview. To help the understanding of thedisclosure, the following description is made in which the first camera401 is executed, but the disclosure is not limited to the description.According to various embodiments, in response to not having ever usedthe first camera 401 formerly, the processor 120 may identify a cameraof a usage history, as the camera that will display the preview. Inresponse to there not being the camera usage history in relation withthe application, the processor 120 may identify the camera that willdisplay the preview, as the first camera 401. In other words, theprocessor 120 may identify the camera that will display the preview,based on the usage history. In response to there not being the usagehistory (e.g., in response to a default value), the processor 120 mayidentify the first camera 401 as the camera that will display thepreview.

In operation 423, the processor 120 may transmit the information aboutthe preview of the first camera 401 to the display device 160. Inresponse to receiving information about an image recognized through thefirst camera 401, the processor 120 may forward the received informationto the display device 160.

In operation 425, the processor 120 may identify a first AR objectrelated with the first object. The processor 120 may identify the firstobject, and may identify the first AR object corresponding to the firstobject. Information about the first object may have been stored in thememory 130 or the server of the application, and the processor 120 mayidentify the first object through a connection with the memory 130 orthe server of the application. The first AR object related with thefirst object may have been previously given (or mapped) and stored inthe memory 130 or the server of the application. The processor 120 mayidentify the first AR object through the connection with the memory 130or the server of the application.

In various embodiments, in response to the first object being a modelindicating an Olympic mark, the first AR object may be an Olympic medalimage. In some embodiments, in response to the first object being amodel indicating an Olympic character, the first AR object may be anOlympic character image. In other some embodiments, in response to thefirst object being a face of a person, the first AR object may be a hatimage displayed at an upper end part of the first object.

In operation 427, the processor 120 may transmit information about thefirst AR object to the display device 160. In response to identifyingthe first AR object related with the first object, the processor 120 maytransmit information for displaying the first AR object to the displaydevice 160. The information transmitted to the display device 160 may beat least one of a shape of the first AR object, a form, a color, or adisplay location.

In operation 431, the processor 120 may process, at a background, theinformation about the preview of the second camera. The processor 120may continuously receive the information about the preview of the secondcamera from the second camera that is a camera in which a preview is notdisplayed. The processor 120 may process, at the background, thereceived information about the preview of the second camera. Operationof processing the information about the preview of the second camera maycorrespond to operation (e.g., operation 425, operation 431) for thepreview of the first camera.

In various embodiments, operation 425 and operation 431 may be performedsimultaneously or in parallel. In response to the second camera notbeing identified as the camera that will display the preview, operation431 may be continuously performed during the execution of theapplication.

In operation 429, the display device 160 may display the first AR objecton the preview of the first camera. The display device 160 may displaythe first AR object on the preview of the first camera, together (or insuperposition). In some embodiments, the first AR object may bedisplayed in association with the first object included in the previewof the first camera. For example, in response to the first object beinga sculpture located in the middle of the preview, the first AR objectmay be displayed around the first object in the form of surrounding thefirst object. In other some embodiments, the first AR object may bedisplayed in a previously given location. For example, the first ARobject may be displayed in the middle of the preview of the firstcamera, regardless of a location of the first object.

In operation 433, the processor 120 may receive (or obtain) an input forconversion into the second camera. The input for conversion into thesecond camera may include inputs of various schemes. In someembodiments, the processor 120 may obtain an input to a preview screenof the first camera. For example, the processor 120 may obtain a user'sinput of touching an icon indicating camera conversion, included in thepreview screen of the first camera. In other some embodiments, theprocessor 120 may recognize a user's voice, and may identify therecognized voice and identify camera conversion or non-conversion. Forexample, though not illustrated, the electronic device 101 may include amicrophone for recognizing a voice (or be connected with themicrophone), and the processor 120 may recognize (or acquire) a user'svoice “Convert the camera” through the microphone. The processor 120 mayidentify the recognized voice, and identify to convert the camera.

In operation 435, the processor 120 may store the information about thefirst AR object. Operation 433 and operation 435 may be performedsimultaneously or in parallel. In some embodiments, the storedinformation may include information about the object that is identifiedto display the first AR object. In other some embodiments, the storedinformation may include information about at least one of a shape of thefirst AR object, a form, a color, a size, or a display location. Inother some embodiments, the stored information may be information abouta keyword indicating the AR object.

In operation 437, the processor 120 may transmit the information aboutthe preview of the second camera 402 to the display device 160.

In operation 441, the processor 120 may identify a second AR objectrelated with the second object, based on the information about the firstAR object. The second AR object may be an AR object which is relatedwith the first AR object and the second object. For example, the firstAR object may be a virtual image indicating an Olympic medal, and thesecond object may be a face of a person. In this case, the second ARobject may be a virtual medal image displayed at a lower end of the faceof the person. In various embodiments, the second AR object may bemapped with the first AR object and the second object and be stored inthe memory 130 or the server of the application. The processor 120 mayidentify the second AR object through the memory 130 or the server ofthe application, based on that the information about the first AR objectand the second object are identified.

In operation 443, the processor 120 may transmit information about thesecond AR object to the display device 160. The second AR object may bedisplayed on the preview of the second camera 402. In variousembodiments, the information about the second AR object may includeinformation about at least one of a size of the second AR object, ashape, a location, or a color. For example, in response to the second ARobject being a medal shape image, the information about the second ARobject may be location information or location data for displaying at alower end which is spaced a specific distance or more apart from theface of the person included in a preview image.

In operation 445, the display device 160 may display the second ARobject on the preview of the second camera. The display device 160 maydisplay, on the preview of the second camera, the second AR objectcorresponding to the information about the second AR object receivedfrom the processor 120. The display device 160 may display the second ARobject, together (or in superposition) with the object included in thepreview of the second camera.

In various embodiments, the electronic device 101 may include a display(or the display device 160), the memory 130 including storedinstructions, and at least one processor. The at least one processor maybe connected with the memory 130 and be configured to execute the storedinstructions so as to identify a first object on a preview acquired fromthe first camera 401, and display a first AR object related with thefirst object on the preview, and in response to obtaining an input forconverting from the first camera 401 to the second camera 402, storeinformation about the first AR object, and display a second AR objectwhich is related with a second object included in a preview of thesecond camera 402 and the information about the first AR object, on thepreview of the second camera 402.

In various embodiments, a method of the electronic device 101 mayinclude identifying a first object on a preview acquired from the firstcamera 401, displaying a first AR object related with the first objecton the preview, and in response to obtaining an input for convertingfrom the first camera 401 to the second camera 402, storing informationabout the first AR object, and displaying a second AR object which isrelated with a second object included in a preview of the second camera402 and the information about the first AR object, on the preview of thesecond camera 402.

FIG. 5 illustrates an example of a flow of an operation of an electronicdevice for displaying an AR object according to various embodiments ofthe disclosure. FIG. 5 shows the operation of the electronic devicesimultaneously executing a plurality of cameras for the sake ofexecution of an application.

In operation 501, the processor 120 may execute the first camera 401 andthe first camera 402. The processor 120 may execute the plurality ofcameras included in the electronic device 101 for the sake of executionof the application. For example, in response to operation of executingan AR based camera application, the processor 120 may execute (or turnon) a rear camera (e.g., the first camera 401) and a front camera (e.g.,the second camera 402).

In operation 503, the processor 120 may identify a recognizable objectof the first camera 401. The processor 120 may acquire an image throughthe first camera 401. The acquired image may be an image correspondingto a scene that a direction of a lens included in the first camera 401faces. The scene may include a plurality of objects (or things), and theprocessor 120 may identify the image corresponding to the scene, therebyidentifying the plurality of objects included in the scene. Theplurality of objects may include at least one recognizable object, andthe processor 120 may identify the plurality of objects, therebyidentifying a recognizable object. The recognizable object may bepreviously given, and information about the recognizable object may havebeen stored in the memory 130 or the server of the application. Theprocessor 120 may identify the recognizable object corresponding to thepreviously stored information of the recognizable object, based on thepreviously stored information of the recognizable object.

In various embodiments, according as the processor 120 is connected withthe memory 130 or the server of the application, the processor 120 mayidentify whether the at least one object included in the plurality ofobjects corresponds to the recognizable object. In response to the atleast one object corresponding to the recognizable object, the processor120 may identify the at least one object as the recognizable object. Forexample, the processor 120 may identify information such as a locationof the at least one object, a size, a shape, a color or the like. Theprocessor 120 may identify whether the identified informationcorresponds to information about the recognizable object previouslystored in the memory 130 or the server of the application.

In various embodiments, in response to there not being the recognizableobject within the image acquired by the first camera 401, the processor120 may repeat or maintain operation 503 until the recognizable objectis identified. To identify the recognizable object, the processor 120may continuously identify an object from the image acquired by the firstcamera.

In operation 505, the processor 120 may display an AR object on apreview of the first camera 401. The processor 120 may set a camera fordisplaying a preview as the first camera. The preview of the firstcamera may be displayed through the display device 160, and therecognizable object and the AR object may be displayed within thepreview of the first camera, together.

In operation 507, the processor 120 may perform background processingfor the second camera 402. The processor 120 may continuously identifyan image acquired through the second camera 402, regardless ofdisplaying or non-displaying of the preview of the second camera 402.The processor 120 may perform the processing of the second camera 402 ina background state. In various embodiments, operation 507 may beperformed simultaneously or in parallel with operation 503 to operation505.

In operation 509, the processor 120 may obtain an input for convertingthe first camera 401 to the second camera 402. For example, theprocessor 120 may obtain a user's input of clicking an icon which isdisplayed for screen conversion on the preview of the first camera 401,thereby changing a camera for preview from the first camera 401 to thesecond camera 402.

In operation 511, the processor 120 may store information about the ARobject displayed on the preview of the first camera 401. The processor210 may store whether the AR object related with the recognizable objectof the first camera indicates which information. In some embodiments,the stored information may include information about an object that isidentified to display the first AR object. In other some embodiments,the stored information may include information about a shape of the ARobject, a form, a color, a size, or a display location. In other someembodiments, the stored information may be information about a keywordindicating the AR object. For example, in response to the AR objectbeing an Olympic medal shape icon, the processor 120 may store a keyword‘Olympic’. For another example, in response to the AR object being theOlympic medal shape icon, the processor 120 may store the informationabout the shape of the AR object, the form, the color, the displaylocation, the size or the like.

In operation 513, the processor 120 may display another AR object on thepreview of the second camera 402, based on the stored information aboutthe AR object and the object recognized through the second camera 402.Information about the another AR object may have been previously storedin the memory 130 or the server. The processor 120 may analyze thestored AR object and the object recognized through the second camera,thereby identifying the another AR object which is related with thestored AR object and the object recognized through the second camera.For example, the processor 120 may obtain a keyword ‘Olympic’ byanalyzing the stored information about the AR object, and may obtain akeyword ‘face’ by analyzing the object recognized through the secondcamera. The processor 120 may identify another AR object which includestwo keywords ‘Olympic’ and ‘face’. Here, the another AR object may be anOlympic medal displayed at a lower end part of the face.

FIG. 6 illustrates another example of an operation of an electronicdevice for displaying an AR object according to various embodiments ofthe disclosure. FIG. 6 shows the operation of the electronic deviceexecuting one camera for the sake of execution of an application.

In operation 601, the processor 120 may execute the first camera. Theprocessor 120 may execute one camera among a plurality of camerasincluded in the electronic device 101 for the sake of execution of anapplication. For example, in response to operation of executing an ARbased camera application, the processor 120 may execute (or turn on) arear camera. The executed camera may be identified by a usage history ofthe application. For example, the processor 120 may store the usagehistory of the application, and may identify that a camera most recentlyused in relation with the application is the rear camera. The processor120 may identify, as the rear camera, a camera that will be executed inresponse to the execution of the application, based on the identifying.In response to there not being the usage history of the application, theelectronic device 101 may basically identify the rear camera as thecamera that will be executed in response to the execution of theapplication.

In operation 603, the processor 120 may identify a recognizable objectof the first camera. The processor 120 may acquire an image through thefirst camera 401, and identify at least one object included in theacquired image. The processor 120 may identify whether the recognizableobject related with the application has been included among at least oneor more objects.

In various embodiments, information about the recognizable object mayhave been previously stored in the memory 130 or the server of theapplication and according as the processor 120 is connected with thememory 130 or the server of the application, the processor 120 mayidentify whether at least one object corresponds to the recognizableobject. In response to the at least one object corresponding to therecognizable object, the processor 120 may identify the at least oneobject as the recognizable object. For example, the processor 120 mayidentify information such as a location of the at least one object, asize, a shape, a color or the like. The processor 120 may identifywhether the identified information corresponds to the information aboutthe recognizable object previously stored in the memory 130 or theserver of the application. In various embodiments, operation 603 maycorrespond to operation 503 of FIG. 5.

In operation 605, the processor 120 may display an AR object on apreview of the first camera 401. Information about the AR object mayhave been previously stored in the memory 130 or the server of theapplication. According as the recognizable object is identified, theprocessor 120 may identify AR object corresponding to the recognizableobject. The identified AR object may be displayed on the preview of thefirst camera. Operation 605 may correspond to operation 505 of FIG. 5.

In operation 607, the processor 120 may obtain an input for convertingthe first camera 401 into the second camera 402. For example, byobtaining a user's input of clicking an icon which is displayed forscreen conversion on the preview of the first camera, the processor 120may change a camera for preview from the first camera 401 to the secondcamera 402. Operation 607 may correspond to operation 509 of FIG. 5. Inoperation 609, the processor 120 may store information about the ARobject displayed on the preview of the first camera 401.

In operation 611, the processor 120 may execute the second camera, toidentify a recognizable another object. In response to receiving theinput for converting the camera of operation 609, the processor 120 mayexecute the second camera 402. The processor 120 may acquire an imagethrough the second camera 402, and may identify a recognizable anotherobject included in the acquired image. The recognizable another objectmay be an object different from the recognizable object of the firstcamera 401. For example, the recognizable object of the first camera 401may be an Olympic sculpture, and the recognizable another object of thesecond camera 402 may be a face. A detailed operation of identifying therecognizable another object may correspond to operation 603. That is,the processor 120 may identify the recognizable another object includedin the image acquired through the second camera 402, in a methodcorresponding to operation 603.

In operation 613, the processor 120 may display another AR object on thepreview of the second camera 402, based on the stored information andthe another object recognized through the second camera 402. Theprocessor 120 may identify the another AR object which is related withthe AR object stored through operation 609 and the another objectrecognized through the second camera 402. According as the another ARobject is identified, the processor 120 may display the another ARobject on the preview of the second camera 402. Operation 613 maycorrespond to operation 513.

FIG. 7 illustrates an example of a UI according to various embodimentsof the disclosure. The processor 120 may provide various UIs 701, 703and 705 related with an application through the display device 160.

In various embodiments, the UI 701 may include a preview 711, a firstobject 709, a first AR object 707, and/or an icon 710-1. The preview 711may be an image acquired through the first camera 401 that turns on byexecution of the application. Here, the first camera 401 may be a rearcamera. The preview 711 may include a plurality of objects 709, 711-1and 711-2. The plurality of objects may be divided into the recognizablefirst object 709 and the non-recognizable objects 711-1 and 711-2. Theprocessor 120 may identify the recognizable first object 709, therebyidentifying the first AR object 707 related with the first object 709.The processor 120 may display the icon 710-1 within the UI 701. The icon710-1 may be for converting a camera related with the preview, and mayhave been previously given based on the application. The icon 710-1 maybe continuously maintained, regardless of a change of the displayedpreview.

In various embodiments, the processor 120 may identify a sculpture inwhich the recognizable object 709 indicates an Olympic mark, therebyidentifying the first AR object 707 that is an AR object related withthe recognizable object 709. The first AR object 707 may be an icon (orimage) having an Olympic medal shape.

In various embodiments, the processor 120 may change a camera fordisplaying the preview into the second camera, based on obtaining auser's input 702 to the icon 710-1 included in the UI 701. According asthe camera for displaying the preview is changed from the first camerato the second camera, the UI 703 may be displayed. The processor 120 maystore information about the first AR object 707 concurrently withchanging the camera for displaying the preview from the first camera tothe second camera, in sequence or regardless of order.

In various embodiments, the user's input 702 may correspond to at leastone of inputs of various schemes. For example, the user's input 702 maybe at least one of a tap, a double tap or a long tap. In someembodiments, the user's input 702 may have been previously given inrelation with the application. In other some embodiments, the user'sinput 702 may be changed through operation of changing the setting ofthe application.

Referring to FIG. 7, the UI 703 may include a preview 715, a secondobject 717, a second AR object 713, and/or an icon 710-2. The preview715 may be an image acquired through the second camera 402 which turnson according as a camera for displaying the preview is changed. Here,the second camera 402 may be a front camera. The processor 120 mayidentify the recognizable second object 717 included in the preview 715.The processor 120 may identify the second AR object 713 which is relatedwith the second object 717 and the first AR object 707, based on thestored information about the first AR object 707. The second AR object713 may be displayed in various schemes. For example, the second ARobject 713 may consist of an Olympic medal image 713-1 related with thefirst AR object and an Olympic character image 713-2. The Olympic medalimage 713-1 and the Olympic character image 713-2 may be displayed inmutually different locations, respectively. The processor 120 maydisplay the icon 710-2 within the UI 703. The icon 710-2 may correspondto the icon 710-1 included in the UI 701. In response to a user's input704 to the icon 710-2 being obtained, the processor 120 may change thecamera for displaying the preview into the first camera. The processor120 may store information about the second AR object 713 concurrentlywith changing the camera for displaying the preview from the secondcamera to the first camera, or regardless of order. The user's input 704may correspond to the user's input 702.

Referring to FIG. 7, the UI 703 may include a preview 723, a thirdobject 721, a third AR object 719, and/or an icon 710-3. The preview 723may be an image acquired through the first camera 401 which turns onaccording as a camera for displaying the preview is changed. Theprocessor 120 may identify the recognizable third object 721 included inthe preview 723. The processor 120 may identify the third AR object 719which is related with the third object 721 and the second AR object 707,based on the stored information about the second AR object 713. Theprocessor 120 may display the icon 710-3 within the UI 703. The icon710-3 may correspond to the icon 710-1 included in the UI 701 or theicon 710-2.

In various embodiments, information for displaying an AR object may beaccumulated, and the processor 120 may display the AR object, based onthe accumulated information. The UI 701, the UI 703, and the UI 705 maybe displayed in sequence, and in this case, the processor 120 maydisplay the UIs, based on information stored in former operations, inthe operation of displaying each of the UIs. According as theinformation used to display the AR object is changed based on theinformation accumulation, another AR object may be displayed accordingto the used information, even though a recognizable object is the same.For example, the third object 721 is the same object as the first object709, but the third AR object 719 may not be the same as the first ARobject 707.

In various embodiments, the camera executed according to the executionof the application may be the second camera. In response to the cameraexecuted according to the execution of the application being the secondcamera, the processor 120 may display the UI 703. In this case, thesecond AR object 713 may be displayed in such a manner that the first ARobject 707 is displayed. In other words, the second AR object 713 may bean AR object related with the second object, which is displayed based onidentifying the second object.

FIG. 8 illustrates another example of a UI for displaying an AR objectaccording to various embodiments of the disclosure.

Referring to FIG. 8, the electronic device 101 may provide various UIs(e.g., UI 801, UI 803, and UI 805) according to execution of anapplication.

In various embodiments, the processor 120 may identify the kind (ortype) of AR for displaying the UI 801. In relation with the application,various AR objects displayable for one object may have been stored inthe memory 130 or a server of the application. Information about thestored various AR objects may be displayed as in a list 803. Six effectsdisplayed in the list 803 may each indicate an effect for showing eachdifferent AR object. For example, the six effects may be effects inwhich colors of the displayed AR objects are each differently set. Foranother example, the six effects may be effects in which locations ofthe displayed AR objects are each differently set. For further example,the six effects may be effects in which the respective displayed ARobjects are all differently set.

In various embodiments, the list 806 may be displayed to show applicableAR effects in relation with an object included in the UI 801. In someembodiments, the list 806 may be an image which is displayed to receivea user's input in response to a camera for displaying the UI 801 beingthe front camera. In other some embodiments, the list 806 may be an ARobject which is displayed based on that the processor 120 recognizes theobject included in the UI 801. In this case, the list 806 may include aplurality of AR objects for showing an applicable plurality of AReffects. The processor 120 may obtain a user's input 807 to one effect805 among the six effects displayed in the list 806. In response toobtaining the user's input 807, the processor 120 may identify the kindof AR object that will be displayed. To identify the kind of AR object,the user's input 807 may be performed in various schemes. For example,the user's input 807 may correspond to at least one of a tap, a doubletap or a long tap. The kind of AR object that will be displayed maycorrespond to the selected effect 805.

In various embodiments, in response to that the user's input 807 to theUI 801 is obtained, the processor 210 may display the UI 803. Forexample, in response to the users' input 807 being obtained in a statewhere the user's face is being displayed on a preview screen through thefront camera, the processor 120 may display an AR object 813. Theprocessor 120 may display the related AR object 813, based on the effect805 and an object 815. For example, the selected effect 805 may be aneffect indicating Olympic, and the object 815 included within a previewacquired through the front camera may be a face. The AR object 813 maybe an Olympic character image 813-1 displayed at an upper end of theobject 815 and an Olympic medal image 813-2 displayed at a lower end ofthe object 815.

In various embodiments, the processor 120 may obtain a user's input 811to the UI 803. The user's input 811 may be a touch on an icon 809included in the UI 803. The icon 809 may be displayed to obtain an inputfor changing a camera related with the UI 803. According as obtainingthe user's input 811 to the icon 809, the processor 120 may change acamera for acquiring a preview in order to display the UI 803, from thefront camera to the rear camera.

In various embodiments, the processor 120 may store information aboutthe existing displayed AR object, based on changing the camera foracquiring the preview. For example, the processor 120 may storeinformation about the AR object 813 that is being displayed, based onobtaining the user's input 811 to the UI 803.

In various embodiments, the processor 120 may display the UI 805, basedon changing the camera for acquiring the preview image. The UI 805 maybe a UI related with a preview acquired through the rear camera. The UI805 may include a recognizable another object 821. The processor 120 maydisplay another AR object 819, based on identifying the another object821. The another AR object 819 may be an AR object which is related withthe another object 821 and the stored AR object 813.

Referring to FIG. 8, an icon 802, the icon 809 and/or an icon 817 mayrefer icons for converting a camera. The icon 802, the icon 809 and/orthe icon 817 may be the same icon for obtaining a user's input in orderto convert the camera.

FIG. 9 illustrates a further example of a UI for displaying an AR objectaccording to various embodiments of the disclosure. FIG. 9 illustratesan example of a UI that is displayed in the electronic device 101 inresponse to a camera displayed by execution of an application being afront camera.

In various embodiments, according as the application is executed, theprocessor 120 may display a UI 901. The UI 901 may include a preview 902acquired through the front camera and an AR object 909. The preview 902may include a recognizable object 905. In response to identifying theobject 905, the processor 120 may identify the AR object 909 fordisplaying in the UI 901. The identified AR object 909 may consist of aplurality of virtual images 909-1 and 909-2. In accordance with variousembodiments, the plurality of virtual images 909-1 and 909-2 may each bethe same image or mutually different images. The identified AR object909 may have been previously given in relation with the object 905, andthe AR object 909 is not limited to the aforementioned example.

In various embodiments, in response to receiving a user's input 911 toan icon 910 displayed within the UI 901, a UI 903 may be displayed. Inresponse to receiving the user's input 911 for converting a camera, theprocessor 120 may store information about the AR object displayed in theUI 901. In some embodiments, the stored information may includeinformation about the object 905 which is identified to display the ARobject. In other some embodiments, the stored information may includeinformation about at least one of a shape of the AR object, a form, acolor, a size, or a location displayed.

In various embodiments, the UI 903 may include a preview 907 acquiredthrough a rear camera and another AR object 913. The preview 907 mayinclude a recognizable object 908. The processor 120 may display theanother AR object 913 within the UI 903, based on the stored informationand the object 908. For example, in response to the object 908 being aplurality of persons, the processor 120 may display the another ARobject 913 in respective face portions of a plurality of persons and/ora given location. The another AR object 913 may include a plurality ofimages. Some (e.g., images 913-3 to 913-7) of the plurality of imagesmay be images corresponding to the object 905, and the remaining some(e.g., 913-1 or 913-2) of the plurality of images may be another imageinto which a shape of the image of the AR object 909 is deformed. Byassociating the AR object displayed after camera conversion and the ARobject before camera conversion, the electronic device 101 may providean abundant, extended experience to a user of the electronic device 101.

In various embodiments, in response to an object recognized through thefront camera being the user, the processor 120 may recognize at leastone of an age of the user, a gender or a look, to change an AR objectthat is displayed by conversion into the rear camera. For example, inresponse to the gender of the user being a female, a color of the ARobject that is being displayed through the rear camera may be changed inorder of red, yellow, and violet.

FIG. 10 illustrates an example of a UI for displaying a recommendationAR object according to various embodiments of the disclosure.

Referring to FIG. 10, a UI 1001 may be a UI which is displayed accordingas a rear camera for displaying a preview is executed in response toexecution of an application. The UI 1001 may include an object 1009 anda plurality of AR objects 1007. The object 1009 may be an object whichis previously given to be recognizable in relation with the application.The plurality of AR objects related with the object 1009 may be given,and in this case, information about the plurality of AR objects may bestored in the memory 130 or a server of the application. By identifyingthe object 1009, the processor 120 may identify the plurality of ARobjects, and display the identified plurality of AR objects 1007.

The plurality of AR objects 1007 may include respective AR objects1007-1, 1007-2 and 1007-3 each having a different attribute or differentimage. The AR object 1007-1 may be displayed in the form of an iconwhich includes an Olympic award certificate shape image. The AR object1007-2 may be displayed in the form of an icon which includes an Olympicmedal shape image. The AR object 1007-3 may be displayed in the form ofan icon which includes a balloon shape image.

The UI 1003 is a UI of displaying a detailed AR object of one AR objectamong the plurality of AR objects. In various embodiments, the processor120 may receive a user's input of selecting one of the plurality of ARobjects displayed in the UI 1001. In response to receiving the user'sinput, the processor 120 may extend an AR object corresponding to theuser's input and display the same. Information about the extended ARobject may have been previously stored in the memory 130 or a server ofthe application. For example, the processor 120 may obtain a user'sinput to the Olympic medal shape AR object 1007-2. In response toobtaining the user's input, the processor 120 may display an extended ARobject 1011. The extended AR object may be an Olympic medal list image.In some embodiments, the Olympic medal list image may be previouslystored in the memory 130 or the server of the application. In other someembodiments, the Olympic medal list image may be real-time refinedthrough the server of the application. The processor 120 may display therefined image, based on being connected to the server of the applicationat a time corresponding to obtaining the user's input.

In various embodiments, the processor 120 may receive a user's input1012 to an icon 1010 included within the UI 1003. The processor 120 mayconvert a camera for displaying a preview, based on receiving the user'sinput 1012. For example, the processor 120 may convert the camera fordisplaying the preview, from the rear camera to the front camera.

Referring to FIG. 10, an AR object 1013 (e.g., 1013-1 and 1013-2) may bedisplayed within a UI 1005. The AR object 1013 may be an AR object whichis related with the extended AR object 1011 and an object 1015 includedin the UI 1005.

In various embodiments, in response to information about the extended ARobject not having been previously given, the extended AR object 1011 maynot be displayed. In this case, the UI 1003 may be omitted, and the UI1005 may be displayed based on receiving a user's input for converting acamera in the UI 1001.

FIG. 11 illustrates an example of an operation of an electronic devicefor displaying a final AR object according to various embodiments of thedisclosure.

According to various embodiments, operation 1101 may correspond tooperation 513 of FIG. 5 or operation 613 of FIG. 6. A duplicateddescription may be omitted in relation with operation 1101 and inrelation with this, may refer to FIG. 5 or FIG. 6. Operation 1103 tooperation 1107 may be performed in sequence after operation 513 oroperation 613 is performed.

In operation 1101, the processor 120 may display the another AR objecton the preview of the second camera 402, based on the stored informationand the another object recognized through the second camera 402.Operation 1101 may be the same or be similar with operation 513 of FIG.5 or operation 613 of FIG. 6.

In operation 1103, the processor 120 may identify whether the displayedanother AR object is a final AR object. In response to there not being afurther AR object that is displayable in relation with the storedinformation in the server of the application or the memory 130, theanother AR object may be identified as the final AR object.

In operation 1105, in response to the displayed another AR object beingthe final AR object, the processor 120 may store information about thefinal AR object. In response to the further AR object displayable havingnot been stored, the processor 120 may identify, as the final AR object,the another AR object displayed through operation 1101. Informationabout the another AR object identified as the final AR object may bestored in the memory 130 or the server of the application.

In operation 1107, the processor 120 may display a notification fornotifying that the information about the final AR object has beenstored. The notification may be displayed in various forms. For example,the processor 120 may show that the information about the final ARobject has been stored through an operation of flickering the final ARobject. For another example, the processor 120 may show that theinformation about the final AR object has been stored by displaying apreviously given notification.

FIG. 12 illustrates an example of a UI for displaying a final AR objectaccording to various embodiments of the disclosure.

In various embodiments, a UI 1201 may correspond to the UI 701 of FIG.7, and a UI 1203 may correspond to the UI 703 of FIG. 7, and a UI 1205may correspond to the UI 705 of FIG. 7. A duplicated operation orconstruction with FIG. 7 may be omitted, and in relation with this, mayrefer to FIG. 7.

A UI 1207 may be a UI which is displayed based on obtaining a user'sinput to an icon for converting a camera displayed in the UI 1205.

Referring to FIG. 12, the UI 1207 may display an AR object, based on theAR object included in the UI 1205 and an object included in a preview ofthe UI 1207. The AR object may be displayed in combination of aplurality of images 1209, 1211, 1213 and 1215. The plurality of imagesmay be the AR object that is configured as one set.

In various embodiments, the AR object displayed in the UI 1207 may bethe final AR object. In response to there not being information relatedwith the AR object displayed in the UI 1207 in the memory 130 or theserver of the application, the processor 120 may identify that the ARobject displayed in the UI 1207 is the final AR object. Though notillustrated, in response to identifying that the AR object displayed inthe UI 1207 is the final AR object, the processor 120 may display anotification. For example, the processor 120 may perform an operation offlickering some of the plurality of images. For another example, todisplay being the final AR object, the processor 120 may repeat anoperation of changing at least one of a color of the plurality ofimages, a brightness, a size, or a location.

FIG. 13 illustrates another example of a UI for displaying a final ARobject according to various embodiments of the disclosure.

In various embodiments, the processor 120 may display a preview byexecuting an application. In a state in which the preview is executed,the electronic device 101 is moved, whereby an image acquired through acamera may be changed. In this case, the processor 120 may identify thechanged image, to refine the preview continuously. In response to apreviously given object 1302 being included within the refined preview,the processor 120 may identify the object 1302, and display an AR object1308 related with this.

Referring to FIG. 13, the AR object 1308 may consist of a plurality ofimages. The plurality of images may be distinguished into a focusedimage (e.g., a stamp) and images (e.g., effects) showing an effect foremphasizing the focused image near the focused image. For example, theAR object 1308 may include images showing effects of triangle forms anda central stamp.

In various embodiments, in response to a record related with the stamphaving been stored in the electronic device 101, the electronic device101 may display the number of the stored records. An icon 1304 displayedwithin a UI 1301 may display the record which is stored in relation withthe stamp. In response to the number of images related with the stampstored in the server of the application being 12, and the number of thestored stamp records being 3, the icon 1304 may be displayed, togetherwith a text ‘3/12’. The text displayed together with the icon 1304 maybe maintained continuously until before the stamp record is changed.

In various embodiments, an icon for converting a camera may be displayedin the form of an icon 1309-1. In response to an input 1309-2 to theicon 1309-1 being obtained, the processor 120 may store informationabout the AR object 1308. The processor 120 may change the camera fordisplaying the preview, based on that the information about the ARobject 1308 is stored. The processor 120 may display a UI 1303, based onan image acquired through the changed camera. The processor 120 mayidentify a recognizable object 1311 included within the image acquiredthrough the changed camera. The processor 120 may display an AR object1310 within the UI 1303, based on the identified object 1311 and thestored information.

In various embodiments, in response to an input 1309-2 for convertingthe camera being obtained, the processor 120 may continuously acquire animage through the changed camera. The processor 120 may continuouslyidentify an object included in the acquired image, and may identify therelated AR object 1310, based on the identifying. For example, accordingas the camera for displaying the preview is changed, the processor 120may acquire a video (or a moving image) for displaying another AR object1310 which is related with the stored information and the object. Here,the UI 1303 may be a part of the acquired video. A UI 1305 indicates aUI of timing at which the acquisition of the video is completed.According as the video acquisition is completed, the processor 120 maydisplay an icon 1306-1 for storing the acquired video within a UI 1305.In response to obtaining a user's input 1306-2 to the displayed icon,the processor 120 may store the acquired video. The processor 120 maydisplay a UI 1307, based on that the video is stored. The UI 1307 mayinclude an image 1313 corresponding to a preview of the stored video. Inresponse to the acquired video being stored, the UI 1307 may refine atext of an icon 1311 displaying a stamp record (or stored stamp). Forexample, the icon 1304 displayed before the stamp record is stored maybe refined into the icon 1312.

FIG. 14 illustrates an example of an operation of the electronic device101 for displaying a recommendation AR object according to variousembodiments of the disclosure.

In operation 1401, the processor 120 may execute the first camera 401.The processor 120 may execute the first camera 401 according as theapplication is executed.

In operation 1403, the processor 120 may identify a recognizable objectthrough the first camera 401. The processor 120 may identify an imageacquired by the first camera 401, thereby identifying a recognizableobject included within the acquired image.

In operation 1405, the processor 120 may display a plurality of ARobjects on a preview of the first camera 401. The processor 120 mayidentify the plurality of AR objects related with the identified object,based on the identified object. The plurality of AR objects related withthe object may be previously given and be stored in the memory 130 or aserver of the application.

In operation 1407, the processor 120 may obtain an input for convertingthe first camera 401 into the second camera 402. The processor 120 maychange a camera for displaying a preview from the first camera 401 tothe second camera 402.

In operation 1409, the processor 120 may store information about theplurality of AR objects. In response to obtaining the input forconverting the camera, the processor 120 may store information about theplurality of AR objects related with the object.

In operation 1411, the processor 120 may execute the second camera 402,and identify a recognizable another object. The processor 120 mayexecute the second camera 402, and may acquire an image through thesecond camera 402. The acquired image may include a plurality ofobjects, and the processor 120 may identify the recognizable anotherobject among the included plurality of objects.

In operation 1413, the processor 120 may identify, as a recommendationAR object, one AR object related with the identified another objectamong the plurality of AR objects. The processor 120 may identifyanother object, thereby acquiring related information from the memory130 or the server of the application. The processor 120 may identify oneAR object having the highest relationship with the identified anotherobject among the plurality of AR objects, based on the acquiredinformation. For example, in response to the identified another objectbeing a user of the electronic device 101, the processor 120 may acquireinformation about a user's preference from the memory 130. The processor120 may identify, as a recommendation AR object, an AR object closest tothe user's preference among the plurality of AR objects.

In operation 1415, the processor 120 may display the recommendation ARobject on a preview of the second camera. In some embodiments, inresponse to the preview of the second camera being displayed, theprocessor 120 may display the recommendation AR object. In other someembodiments, the processor 120 may display another AR object forrepresenting the recommendation AR object. In response to a user's inputto the another AR object being obtained, the processor 120 may displaythe recommendation AR object. In further other some embodiments, theprocessor 120 may preferentially display a part of the recommendation ARobject. In response to a user's input to the part of the recommendationAR object being obtained, the recommendation AR object may be alldisplayed.

FIG. 15 illustrates another example of an operation of the electronicdevice 101 for displaying a recommendation AR object according tovarious embodiments of the disclosure.

In operation 1501, the processor 120 may execute the first camera 401and the second camera 402. According as an application is executed, theprocessor 120 may execute the first camera 401 and the second camera402.

In operation 1503, the processor 120 may identify a recognizable objectthrough the first camera 401. The processor 120 may identify an imageacquired by the first camera 401, thereby identifying the recognizableobject included within the acquired image.

In operation 1505, the processor 120 may display a plurality of ARobjects on a preview of the first camera 401. The processor 120 mayidentify the plurality of AR objects related with the identified object,based on the identified object. The plurality of AR objects related withthe object may be previously given and be stored in the memory 130 or aserver of the application.

In operation 1507, the processor 120 may process the second camera 402at a background. While the preview of the first camera 401 is displayed,the processor 120 may identify an image through the second camera 402 atthe background. A recognizable another object may be included within theimage acquired through the second camera 402. The processor 120 mayidentify the image acquired through the second camera 402, therebyidentifying the recognizable another object.

In various embodiments, in response to operation 1501 being performed,operation 1507 may be continuously performed.

In operation 1509, the processor 120 may obtain an input for convertingthe first camera 401 into the second camera 402. The processor 120 maychange a camera for displaying a preview from the first camera 401 tothe second camera 402.

In operation 1511, the processor 120 may store information about arecommendation AR object among the plurality of AR objects. Theprocessor 120 may identify the recommendation AR object related with theanother object among the plurality of AR objects, based on thatoperation 1507 is continuously maintained. The processor 120 mayidentify, as the recommendation AR object, one AR object having thehighest relationship with the another object recognized through thesecond camera 402 among the plurality of AR objects.

In operation 1513, the processor 120 may display the recommendation ARobject on a preview of the second camera 402. The processor 120 maydisplay the recommendation AR object identified in operation 1511,together with the preview acquired through the second camera 402.

FIG. 16 illustrates an example of an operation of a UI for displaying arecommendation AR object according to various embodiments of thedisclosure.

A UI 1601 may include a plurality of AR objects 1609 related with anobject 1607. In some embodiments, the plurality of AR objects mayinclude an AR object 1609-1 related with a certificate of award, an ARobject 1609-2 related with a balloon, and an AR object 1609-3 relatedwith Olympic. In various embodiments, in response to another objectbeing a face of a user of the electronic device 101, the electronicdevice 101 may link with a server of a social network service (SNS)account of the user, thereby obtaining a user's interest. In response tothe obtained interest is Olympic, the electronic device 101 may identifythe AR object related with Olympic as a recommendation AR object. Inother some embodiments, information about an AR object related with anapplication may be refined continuously through a server of theapplication. The processor 120 may receive information related withpreferences or popularities of various users for the plurality of ARobjects from the server of the application. The processor 120 mayidentify the recommendation AR object, based on the receivedinformation. For example, the processor 120 may identify the AR object1609-3 having the highest preference among the plurality of AR objects1609, as an AR object for displaying the recommendation AR object.

In various embodiments, the plurality of AR objects 1609 included in theUI 1601 may be AR objects which are displayed in the form of an icon inorder to indicate the recommendation AR object, respectively. Forexample, to indicate that the recommendation AR object is the AR objectrelated with Olympic, the AR object 1609-3 of an Olympic icon shape maybe displayed.

In various embodiments, according as a user's input 1611 to an icon1610-1 included in the UI 1601 being obtained, the processor 120 maydisplay a UI 1603. The icon 1610-1 may be an icon which is set toreceive an input for converting a camera, and icons 1610-1 to 1610-3 maybe the same icon. The processor 120 may convert a camera for displayinga preview, based on the user's input to the icon 1610-1.

The UI 1603 may include a recommendation AR object. In some embodiments,according as the user's input 1611 for converting the camera beingobtained in the UI 1301, the processor 120 may display the UI 1603including the recommendation AR object. The UI 1603 may include anotherAR object 1613 for indicating the recommendation AR object. The anotherAR object 1613 may include an AR object 1613-1 of a text box form, an ARobject 1613-3 of a flag shape, and an AR object 1613-2 configured toreceive an input. In response to a user's input to the AR object 1613-2being obtained, a UI 1605 may be displayed. The UI 1605 may display arecommendation AR object 1615 which is related with an object recognizedthrough the camera and the user's input. Thought not illustrated, theanother AR object 1613 may consist of a part of the recommendation ARobject. In response to a user's input to the part of the recommendationAR object 1615 being obtained, the recommendation AR object 1615 may beall displayed. The recommendation AR object 1615 may be displayed invarious schemes. For example, the recommendation AR object 1615 mayinclude an Olympic medal image 1615-2 and an Olympic character image1615-1.

FIG. 17 illustrates an example of a signal flow of an electronic devicefor displaying an AR object related with another electronic deviceaccording to various embodiments of the disclosure.

In operation 1701, the electronic device 101 may be connected with theanother electronic device 1700. The electronic device 101 may beconnected with the another electronic device 1700, based on that anapplication is executed. Because an operation (e.g., pairing) ofconnecting the electronic device 101 and the another electronic device1700 with each other corresponds to the technology of the related art, adetailed description related with this may be omitted. The electronicdevice 101 may be connected with the another electronic device 1700,thereby transmitting and/or receiving various information with theanother electronic device 1700. For example, the electronic device 101and the another electronic device 1700 may transmit and/or receiveinformation about an image acquired through a camera included each ofthem.

In operation 1702, the electronic device 101 may recognize a previewincluding a first object through the first camera. The electronic device101 may identify the first object included within the image acquiredthrough the first camera.

In operation 1704, the electronic device 101 may identify a first ARobject related with the first object. The electronic device 101 mayidentify the first AR object related with the identified first object.Information about the first AR object may have been stored in the serverof the application or the memory 130.

In operation 1706, the electronic device 101 may display the first ARobject on the preview of the first camera. For example, the electronicdevice 101 may display the first AR object such that the first AR objectis located around the first object included in the preview of the firstcamera.

In operation 1708, the electronic device 101 may obtain an input relatedwith the second camera. The electronic device 101 may obtain a user'sinput related with the first AR object included in the displayedpreview. In various embodiments, the electronic device 101 may receive auser's input related with the displayed preview. In some embodiments,the electronic device 101 may receive a user's input for storinginformation about the first AR object included in the displayed preview.In some embodiments, the electronic device 101 may receive a user'sinput for transmitting the information about the first object includedin the displayed preview to another electronic device. The user's inputmay be, for example, an input to at least one icon displayed on thepreview. For another example, the user's input may be a voice signalwhich is recognized based on a voice recognition function.

In operation 1703, the another electronic device 1700 may recognize apreview including a second object through the second camera. The secondcamera may be a camera included in the another electronic device. Theelectronic device 101 may receive an image acquired through the secondcamera of the another electronic device 1700 from the another electronicdevice 1700, and identify the second object included within the receivedimage. In various embodiments, the second camera may be a camera that isdirectly or indirectly connected with the electronic device 101. In someembodiments, the electronic device 101 may identify information aboutthe image acquired from the second camera through the processor 120,based on a connection with the another electronic device 1700. Operation1703 may be performed at the same time of performing operation 1702 tooperation 1706.

In operation 1705, the another electronic device 1700 may display thepreview including the second object through the second camera. Invarious embodiments, operation 1705 may be omitted, and informationabout the preview of the second camera may be transmitted to theelectronic device 101 and be identified through the processor 120.

In operation 1710, the electronic device 101 may transmit informationabout the first AR object which is related with the input obtainedthrough operation 1708, to the another electronic device 1700. Accordingas obtaining the user's input related with the second camera, theelectronic device 101 may store the information about the first ARobject in the another electronic device 1700.

In operation 1711, the another electronic device 1700 may store theinformation about the first AR object, based on receiving theinformation about the first AR object. The another electronic device1700 may store the information about the first AR object, based onreceiving a signal on the input of the electronic device 101 relatedwith the second camera. For example, while a video call connection withthe electronic device 101 is maintained, the another electronic device1700 may receive a voice signal of the first AR object from theelectronic device 101. According as the voice signal is received, theanother electronic device 1700 may store the information about the firstAR object.

Thought not illustrated, in various embodiments, the electronic device101 and the another electronic device 1700 may share (or transmit and/orreceive, or synchronize) information about each preview at a giveninterval of time, based on operation 1701. In this case, according asoperation 1710 is performed, the another electronic device 1700 mayidentify the first AR object for which the user's input is obtainedamong the shared information. The another electronic device 1700 maystore the information about the first AR object, based on theidentifying of the first AR object.

In operation 1713, the another electronic device 1700 may identify asecond AR object related with the second object, based on theinformation about the first AR object. The another electronic device1700 may identify the second AR object included within the imageacquired through the second camera. The another electronic device 1700may identify the second AR object, based on the identified second objectand the stored information about the first AR object.

In operation 1715, the another electronic device 1700 may display thesecond AR object on the preview of the second camera. The anotherelectronic device 1700 may display the second AR object identifiedthrough operation 1713, on the preview of the second camera.

In various embodiments, the another electronic device 1700 and theelectronic device 101 may operate reversely. For example, the electronicdevice 101 may perform an operation of the another electronic device1700 of FIG. 17, and the another electronic device 1700 may perform anoperation of the electronic device 101.

FIG. 18 illustrates an example of a UI of an electronic device fordisplaying an AR object related with another electronic device accordingto various embodiments of the disclosure.

In various embodiments, the electronic device 101 may be wiredly orwirelessly connected with another electronic device. A UI 1801 may bedisplayed in the electronic device 101, based on the connection, and aUI 1803 may be displayed in the another electronic device connected withthe electronic device 101.

The UI 1801 may include a preview acquired through a front cameraincluded in the electronic device 101. The electronic device 101 mayidentify an object 1809 included in the UI 1801, and may display atleast one first AR object 1811-1 or 1811-2, based on the identifying.The electronic device 101 may identify and display an AR objectcorresponding to interest information. For example, by identifying thatthe object 1809 is a user of the electronic device 101, the electronicdevice 101 may acquire the interest information stored in a SNS accountof the user. The interest information may be information which is readmost recently by the user or is marked to be of interest (e.g., whosegiven icon is clicked). For another example, by identifying that theobject 1809 is the user of the electronic device 101, the electronicdevice 101 may acquire information stored in a cloud service of theuser. The electronic device 101 may identify at least one of theacquired information, as user's interest information. For furtherexample, the interest information may be identified based on a searchhistory stored in the electronic device 101, screen shot (or storedimage) information, etc.

The UI 1803 may include a preview acquired through a front cameraincluded in the another electronic device. The another electronic devicemay identify an object 1810 included in the UI 1803, and may display atleast one second AR object 1812-1, 1812-2 or 1812-3, based on theidentifying of the object 1810.

In various embodiments, at least one first AR object 1811-1 or 1811-2may be displayed in various forms, without being limited to thedisclosure in relation with the object 1809. The form in which the atleast one second AR object 1812-1, 1812-2 or 1812-3 is displayed is notlimited to the disclosure.

In various embodiments, the electronic device 101 may receive a giveninput to the UI 1801. The given input may be an input for sharing, bythe electronic device 101, information with the another electronicdevice. In response to receiving the given input, the electronic device101 may share the information with the another electronic device.

In various embodiments, the another electronic device may display a UI1807, based on the shared information. The UI 1807 may be a UI which isdisplayed through the another electronic device in response to the giveninput being received from the electronic device 101. The UI 1807 mayinclude an AR object 1815 related with the object 1817 or an AR object1817 related with the shared information. For example, AR objects1815-1, 1815-2, 1815-3 and 1815-4 may associated with the object 1817.

In various embodiments, the electronic device 101 may display a UI 1805,based on the shared information. The UI 1805 may identify an objectrecognized through the electronic device 101, and display a refined ARobject 1813. For example, in response to an object included in thepreview acquired through the electronic device 101 being a male, inresponse to the object being identified, the electronic device 101 maydisplay an AR object 1813-1 showing being the male. For another example,in response to the object included in the preview acquired through theelectronic device 101 being the male, in response to the object beingidentified, the electronic device 101 may display the AR object 1813-1showing being the male. For further example, in response to the receivedinput being an input for sharing information about a song, theelectronic device 101 may display an AR object 1813-2 or 1813-3 forindicating the information about the song.

FIG. 19 illustrates another example of a UI of an electronic device fordisplaying an AR object related with another electronic device accordingto various embodiments of the disclosure.

In various embodiments, FIG. 19 illustrates an example of a UI which isdisplayed according as the electronic device 101 (e.g., a device of A)and another electronic device (e.g., a device of B) are connected witheach other. The electronic device 101 and the another electronic devicemay be devices capable of performing the same function or operation.

In various embodiments, the electronic device 101 and the anotherelectronic device may be wiredly or wirelessly connected with eachother. For example, according as a video call is executed, theelectronic device 101 and the another electronic device may bewirelessly connected with each other. In response to the connectionbeing established, the electronic device 101 may display a UI 1901, andthe another electronic device may display a UI 1903.

The UI 1901 may include a recognizable object 1909 of the electronicdevice 101 and an AR object 1910 related with the object. The electronicdevice 101 may previously select the object 1909, based on informationstored by a user of the electronic device 101. The stored informationmay be, for example, information about an appointment place which isstored by the user of the electronic device 101.

In various embodiments, the another electronic device may display the UI1903, based on a connection with the electronic device 101. The anotherelectronic device may change the UI 1903 into a UI 1905, based onreceiving a given input. For example, the another electronic device maydisplay the UI 1905 related with the electronic device 101, based onobtaining a given voice signal such as ‘where are you?’. In response toreceiving the given input, the another electronic device may storeinformation about the AR object 1910.

The UI 1905 may include a preview acquired through a rear camera of theanother electronic device. The another electronic device may display anAR object 1906, based on the information about the AR object 1910 and anobject identified through the another electronic device. The AR object1906 may be information related with the AR object 1910 displayedthrough the electronic device 101. For example, the AR object 1910 andthe AR object 1906 may be mutually different AR objects for displayingthe same place. The object identified through the another electronicdevice may be at least one thing included in the preview. For example,the identified object may be a signpost indicating a location includedin the preview. The another electronic device may identify a location ofthe another electronic device by identifying the signpost, and maydisplay the AR object 1906, based on the identified location and the ARobject 1910.

The electronic device 101 may receive information about the AR object1906 included in the UI 1905, based on the connection with the anotherelectronic device. The electronic device 101 may display a UI 1907,based on the received information. The UI 1907 may include an AR object1911 showing a location of a user of the another electronic device. TheAR object 1911 showing the location of the user may include, forexample, the text ‘B is now at exit 2 of Yangjae station’.

According to an electronic device and an operation method thereofaccording to various embodiments, the electronic device may provide,together, an AR object associated with an image acquired through a firstcamera and another AR object associated with an image acquired through asecond camera, thereby providing an extended experience to a user.

According to various embodiments, the electronic device may storeinformation about an AR object related with an object acquired throughthe first camera.

According to various embodiments, the electronic device may display theAR object of the acquired object, based on the stored information aboutthe AR object, thereby providing a wide experience to the user.

An effect that may be obtained from the disclosure is not limited to theeffects mentioned above, and other effects not mentioned would be ableto be apparently understood from the following statement by a personhaving ordinary skill in the art to which the disclosure pertains.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a display; amemory comprising instructions stored thereon; and at least oneprocessor coupled with the memory and the display, wherein the at leastone processor is configured to execute the stored instructions so as to:identify a first object on a preview acquired from a first camera,display a first augmented reality (AR) object related with the firstobject on the preview, in response to obtaining an input for convertingfrom the first camera to a second camera, store information about thefirst AR object, and display a second AR object which is related with asecond object comprised in a preview of the second camera and theinformation about the first AR object, on the preview of the secondcamera.
 2. The electronic device of claim 1, wherein the at least oneprocessor is further configured to execute the stored instructions so asto: in response to obtaining an input of converting from the secondcamera to the first camera, store information about the second ARobject; and display a third AR object which is related with a thirdobject comprised in the preview of the first camera and the storedinformation about the second AR object on the preview of the firstcamera, based on the stored information about the second AR object. 3.The electronic device of claim 2, wherein the third object correspondsto the first object.
 4. The electronic device of claim 1, wherein the atleast one processor is further configured to execute the storedinstructions so as to display a plurality of AR objects on a previewimage of the first camera, based on comprising the plurality of ARobjects related with the first object.
 5. The electronic device of claim4, wherein the at least one processor is further configured to executethe stored instruction so as to, in response to identifying an input forselecting one of the plurality of AR objects, identify the selected ARobject as the first AR object.
 6. The electronic device of claim 4,wherein the at least one processor is further configured to execute thestored instructions so as to: in response to obtaining the input forconverting from the first camera to the second camera, identify an ARobject related with a user of the electronic device among the pluralityof AR objects; store information about the identified AR object; anddisplay the identified AR object on the preview of the second camera. 7.The electronic device of claim 4, wherein the at least one processor isfurther configured to execute the stored instructions so as to: inresponse to obtaining the input of converting from the first camera tothe second camera, identify an AR object related with a user of theelectronic device among the plurality of AR objects; store informationabout the identified AR object, and display a third AR object forindicating the identified AR object on the preview of the second camera;and in response to obtaining an input to the third AR object, displaythe identified AR object on the preview of the second camera.
 8. Theelectronic device of claim 1, wherein the at least one processor isfurther configured to execute the stored instructions so as to: inresponse to obtaining a user's input for execution of an applicationrelated with the first AR object and the second AR object, execute thefirst camera and the second camera; while the preview of the secondcamera is displayed, refine the first AR object through the firstcamera; store information about the refined first AR object; and refinethe second AR object, based on the stored information.
 9. The electronicdevice of claim 1, wherein the at least one processor is furtherconfigured to execute the stored instructions so as to: identify whethera plurality of cameras comprised in the electronic device have beenformerly used; in response to at least one camera among the plurality ofcameras has been formerly used, identify, as the first camera, the mostrecently used camera among the plurality of cameras; and in response tothe at least one camera among the plurality of cameras has not beenformerly used, identify, as the first camera, a camera disposed in arear surface of the electronic device.
 10. The electronic device ofclaim 1, wherein the first camera comprises a camera comprised inanother electronic device, based on that the electronic device iswiredly or wirelessly connected with the other electronic device, andwherein the first AR object is related with an object identified throughthe camera comprised in the other electronic device.
 11. A method for anelectronic device, comprising: identifying a first object on a previewacquired from a first camera; displaying a first augmented reality (AR)object related with the first object on the preview; in response toobtaining an input for converting from the first camera to a secondcamera, storing information about the first AR object; and displaying asecond AR object which is related with a second object comprised in apreview of the second camera and the information about the first ARobject, on the preview of the second camera.
 12. The method of claim 11,further comprising: in response to obtaining an input of converting fromthe second camera to the first camera, storing information about thesecond AR object; and displaying a third AR object which is related witha third object comprised in the preview of the first camera and thestored information about the second AR object on the preview of thefirst camera, based on the stored information about the second ARobject.
 13. The method of claim 12, wherein the third object correspondsto the first object.
 14. The method of claim 11, further comprisingdisplaying a plurality of AR objects on a preview image of the firstcamera, based on comprising the plurality of AR objects related with thefirst object.
 15. The method of claim 14, further comprising, inresponse to identifying an input for selecting one of the plurality ofAR objects, identifying the selected AR object as the first AR object.16. The method of claim 14, further comprising: in response to obtainingthe input for converting from the first camera to the second camera,identifying an AR object related with a user of the electronic deviceamong the plurality of AR objects; storing information about theidentified AR object; and displaying the identified AR object on thepreview of the second camera.
 17. The method of claim 14, furthercomprising: in response to obtaining the input of converting from thefirst camera to the second camera, identifying an AR object related witha user of the electronic device among the plurality of AR objects;storing information about the identified AR object; displaying a thirdAR object for indicating the identified AR object on the preview of thesecond camera; and in response to obtaining an input to the third ARobject, displaying the identified AR object on the preview of the secondcamera.
 18. The method of claim 11, further comprising: in response toobtaining a user's input for execution of an application related withthe first AR object and the second AR object, executing the first cameraand the second camera; while the preview of the second camera isdisplayed, refining the first AR object through the first camera;storing information about the refined first AR object; and refining thesecond AR object, based on the stored information.
 19. The method ofclaim 11, further comprising: identifying whether a plurality of camerascomprised in the electronic device have been formerly used; in responseto at least one camera among the plurality of cameras has been formerlyused, identifying, as the first camera, the most recently used cameraamong the plurality of cameras; and in response to the at least onecamera among the plurality of cameras has not been formerly used,identifying, as the first camera, a camera disposed in a rear surface ofthe electronic device.
 20. The method of claim 11, wherein the firstcamera comprises a camera comprised in another electronic device, basedon that the electronic device is wiredly or wirelessly connected withthe other electronic device, and wherein the first AR object is relatedwith an object identified through the camera comprised in the otherelectronic device.